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Abstract: This paper develops some objective priors for certain parameters of 
the bivariate normal distribution. The parameters considered are the regression 
coefficient, the generalized variance, and the ratio of the conditional variance of 
one variable given the other to the marginal variance of the other variable. The 
criterion used is the asymptotic matching of coverage probabilities of Bayesian 
credible intervals with the corresponding frequentist coverage probabilities. 
The paper uses various matching criteria, namely, quantile matching, matching 
of distribution functions, highest posterior density matching, and matching via 
inversion of test statistics. One particular prior is found which meets all the 
matching criteria individually for all the parameters of interest. 
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1. Introduction 

Bayesian methods are becoming increasingly popular in the theory and practice of 
statistics. It is needless to say that other than the likelihood, the key component in 
any Bayesian analysis is the selection of priors. With sufficient information from past 
experience, expert opinion or previously collected data, subjective priors are ideal, 
and indeed should be used for inferential purposes. However, often even without 
adequate prior information, one can use Bayesian techniques efficiently with some 
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"default" or "objective" priors. Not too surprisingly, the catalog of such priors, over 
the years, has also become prohibitively large, and one needs to set some specific 
"objectivity" criterion for the solution of such priors. 

One such criterion which has found some appeal to both frequentists and 
Bayesians is the so-called "probability matching" criterion. Simply put, this 
amounts to the requirement that the coverage probability of a Bayesian credible 
region is asymptotically equivalent to the coverage probability of the frequcntist 
confidence region up to a certain order. An excellent monograph on this topic is 
due to Datta and Mukerjee [-5] which provides a thorough and comprehensive dis- 
cussion of various probability matching criteria. Other review papers include [(i], 
[7] and [<-)]. 

Again, as one might expect, there are several probability matching criteria. The 
matching is accomplished through either (a) posterior quantiles, (b) distribution 
functions, (c) highest posterior density (HPD) regions, or (d) inversion of certain 
test statistics. However, priors based on (a), (b), (c), or (d) need not always be 
identical. Specifically, it may so happen that there does not exist any prior satisfying 
all four criteria. 

In this article, we consider the bivariatc normal distribution where the parame- 
ters of interest are the (i) regression coefficient, (ii) the generalized variance, i.e. the 
determinant of the variance-covariance matrix, and (iii) the ratio of the conditional 
variance of one variable given the other divided by the marginal variance of the 
other variable. We have been able to find a prior which meets all four matching 
criteria for every one of these parameters. 

As is well known, matching priors are obtained by solving certain differential 
equations. In Section 2 of this paper, we have introduced an orthogonal reparam- 
eterization ([3], [S]) of the bivariate normal parameters which greatly simplifies 
these equations, resulting thereby in easily accessible solutions. Here, we have also 
introduced a quantilc matching prior which works for all the parameters given in 
(i)-(iii). Section 3 establishes the distribution matching property of the prior found 
in Section 2, once again for all three parameters of interest. Section 4 establishes 
the HPD matching property of such priors, while Section 5 confirms matching by 
inversion of the likelihood ratio statistic. The propriety of the posteriors and some 
numerical computations are given in Section 6, while some final remarks are made 
in Section 7. 

2. The orthogonal reparameterization 

Let {Xu^X2i),i = l,...,n, be independent and identically distributed random 
variables having a bivariate normal distribution with means and ^2, variances 
(Ti^(> 0) and (T2^(> 0), and correlation coefficient p{\p\ < 1). We use the transfor- 
mation (see, e.g., [1], [4] or [10]) 

(2.1) P = pa2/ai, = <7ia2(l - p^f^ and 7? = ^2(1 - p^f^/n. 

With this reparameterization, the bivariate normal distribution can be rewritten as 

where (3 is the regression coefficient of X2 on ATi, 0^ = o-i^(T2^(1 ^ P^) is the 
determinant of the variance-covariance matrix, and 77^ = V(X2\Xi)/V(Xi). 
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With the above reparameterization, the Fisher Information matrix reduces to 

A 

Diag(77'^ 6l~2, ry- 



A 

(2-3) ^l2, 13, 0, 1^) = [ _ r>:„„/^-2 /j-2 ^-2 



where 



\ 13 1 



\ 6»? Sr] / 

This estabhshes immediately the mutual orthogonality of {^i, ^2), d and ?/ in 
the sense of [■'■>] and [8] . Such orthogonality is often referred to as "Fisher Orthogo- 
nality." 

Since the parameters of interest are orthogonal to (/ii,/i2), and it is customary 
to use a uniform (5R^) prior on (/ii, ^2), we shall consider only priors of the form 

(2.4) 7ro(Aii,/i2,/3,6',?/) cx 7r(/3,6',77), 

and find tt such that the matching criteria given in (a)-(d) are all satisfied for /?, 9 
and 1], each individually. This we are going to explore in the next three sections. 

Here we also state a lemma which is used repeatedly in the sequel. The proof is 
based on the independence of X2 — /i2 ^ l3{Xi — /ii) and Xi — /ii along with the 
fact that X2 - ^-L2 - (3{Xi - m) ~ iV(0, 9rj) and Xi - m ^ N{0, O/ifj. 

Lemma 2.1. For the bivariate normal density given in (2.2), 

(2.5) E{d log f/dpf = Q, E[{d log f/dp) [d^ log f/dp^)] = 0; 
(2.6) 

E{d'^\ogf/dp'^) = 0, E{d^\ogf/dp'^d9) = {07]"^) \ E{d'^\ogf /dp'^drf) = 77-^; 

(2.7) E{d^\ogf/dpde^) = 0,E{d^\ogf /dpdri'^) = 0; 

(2.8) E{d\ogf/def = 2/6'^ E[{d\ogf/de){d^\ogf/de^)] = -2/6i3; 
(2.9) 

E{d^\ogf/de^) ^ 4/6*3, E{d^logf/de^dT]) = 0, Eid"^ log f / dOdif) = {Oif) ^ 



.2 E{d\ogf/dr^f 

^' ' E[(d\ogf /dr]){d^ log f/drf)] = -rj-^ , E{dHog f /d7f) ^ Sr]'^ 



We consider in this section quantilc matching priors, i.e., priors which ensure 
approximate frequentist validity of one-sided Bayesian credible intervals based on 
posterior quantiles of a one-dimensional interest parameter. The pioneering research 
in this area is due to Welch and Peers [14] and Peers [11], while the recent stimulus 
comes from Stein [12] and Tibshirani [J 'i]. Specifically, one considers here priors 
7r(-) for which the relation 

(2.11) P0{0i < 6'i^""(7r, X)} = l-a + o{n~i) 

holds for r = 1 or 2 and for each a (0 < a < 1). In the above, 9 = (6*1 , ... , 6p)'^ is the 
unknown parameter, 61 is the one-dimensional parameter of interest, 9i^~°'{t:, X) 
is the (1 — a)th posterior quantilc of 9i based on the prior tt and data X = 



74 



M. Ghosh, U. Santra and D. Kim 



{Xi, . . . ,Xn)'^, while P{-\d) denotes the conditional probability given 9, the usual 
frequentist probability. A prior satisfying (2.11) with r = 1 is called a first order 
probability matching prior, while one with r = 2 is called a second order probabil- 
ity matching prior. Clearly, second order probability matching priors constitute a 
subclass of first order probability matching priors. 

A second order quantile matching prior which works for (3, 6 and rj is given by 
Tr{^i, ^2, [3,0,7]) oc {01]) . Back to the original parameterization, (/ii, yU2, ci, (T2, 
p), this reduces to the prior 

(2.12) 7r(^i,^2,0-i,cr2,p) = cri"^(l - p^) \ 

This prior has been identified in [2] as the right-Haar prior as well as the one-at-a- 
time reference prior for any arbitrary ordering of these parameters. Indeed, this prior 
provides exact rather than just asymptotic matching for a variety of parameters of 
interest including the ones considered here. Moreover, as shown in this paper, when 
(3 is the parameter of interest, any prior of the form o'i~'^~°'(l — p^) ^ , a > 0, for 
(Mi j M2i en I o'2! p) achieves exact matching, while when 6 is the parameter of interest, 
both the priors cri^^(l — p^) ^ and <ti^^<T2^^{1 — fP') '^^^ achieve exact matching. 

However, Berger and Sun [1] have not explored other matching criteria. While 
the quantile matching property is quite desirable for construction of one-sided cred- 
ible intervals, the HPD matching or matching via inversion of test statistics seems 
more appropriate for the construction of two-sided credible intervals. It is also true 
that priors which satisfy a particular matching criterion, for example, the quantile 
matching criterion, satisfies other matching criteria. As mentioned in the introduc- 
tion, we will explore various other matching criteria in the subsequent sections. In 
this way, we will lend further justification to one of the matching priors of [2] - the 
one that we have developed here, as well. 

3. Matching via distribution functions 

In this section, we target priors tt which achieve matching via distribution functions 
of some standardized variables. More specifically, when 9i is the parameter of in- 
terest, (02, ■ ■ • , Op)"^ is the vector of nuisance parameters, 9i is the MLE of 9i with 
n~il^^ (/ = {{Ijji)),I~^ = {{P^ ))) as its asymptotic variance, we consider the 

1 /2 

random variable y = \/n.(9i — 0i)/{I^^) . Specifically, if P'^ denotes the posterior 
of y given the data X, what we want to achieve is the asymptotic matching 

(3.1) ElP^'iy < w\X)\9] = P{y < w\9) + o{n-^). 

Under orthogonality of 9i with (6*2, ... , 9p), it follows from (3.2.5) to (3.2.7) of [5] 

1/2 

that such a prior tt is of the form ff(^2: ■ • ■ , Op), where in addition one needs to 
satisfy the two differential equations 

(3.2) 

s—2 v—2 ^ 
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In the given problem, when f3 is the parameter of interest, (3.3) reduces to 

By (2.6) of Lemma 2.1, (3.4) simplifies to d/dl3{{9 + r/)7r(/3, 9, ?;)} = which holds 
trivially for any prior 7r(/3, 9, if) which does not depend on /3, including the prior 
'!T{p,9,r]) oc {9ri) , the one found in Section 2. Again, with /3 as the parameter 
of interest, for any prior 7r(/3, 9, ?;) which does not depend on /3, (3.2) simplifies to 
d/d9{{9r]^y\f9^TT{/3, 9, 77)} + d/dT]{f]-^r]^ifTT{l3, 9, 77)} = 0, i.e., 

^[9 7riP,9,7^)] + -^^[rjniP,9,7^)]=0. 

Once again TT{f3,9,ri) cx {9i])^^ will satisfy (3.3). To verify that a prior 7r{(3,9,ri) 
which does not depend on (3 satisfies (3.2) with 9i = /?, we need to verify that 

which reduces to 

2/)2 ^ T^/^^log/- 4 9 log/- 

From (2.7) of Lemma 2.1, £(9^ log //9/396l2) = £(9^ log //9/39r;2) = 0, so that 
(3.2) holds trivially for such a prior. Hence matching via distributions is achieved 
with any prior of the form ■k{^i, ^2, P,9,ri) oc h(9,'r]), and in particular h{9,vi) cx 

Next, when 9 is the parameter of interest, to find a prior satisfying (3.1) one 
needs to first solve 



(3.6) 



Hence, (3.6) simplifies to 

Any prior 7r(-) oc 9~^g{[3, rj) will satisfy this equation. Such a prior also satisfies 
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Finally when 77 is the parameter of interest, again, for finding a prior satisfying 
(3.1), one needs to solve 



(3.8) 



and 



Again, by Lemma 2.1, the prior tt{(3, 6,1]) = 6 ^rj ^ satisfies the matching prop- 
erty. 



4. Highest posterior density (HPD) matching priors 

We now turn our attention to HPD matching priors for each one of the parameters 
/3, 6 and rj. In general, if 9 is the parameter (real or vector-valued) of interest, then 
a HPD region is of the form {9 : tt{9\X) > k}, where tt{9\X) is the posterior of 
9 under the prior tt and data X. We will consider priors which ensure that HPD 
regions with credibility level I — a also have asymptotically the same frcquentist 
coverage probability, the error of approximation being o(n~^). 

We first consider a HPD region for (3. In view of the orthogonality result derived 
in Section 2, such a prior 7ro(/^i, fj.2, /3, 9, if) oc 7r(/3, 0, 77) must satisfy (see [o], (4.4.1)) 





|3 log 




Again, by Lemma 2.1. (4.1) reduces to 



(4-2) ^(^"(•)) + |^(^"(-))~^("'"(-))=0- 

Clearly the prior 7r(/3, 0,77) oc {9ri) ^ satisfies (4.2). 

Next consider 9 as the parameter of interest. Now one needs to solve 



(4.3) 




rd^ log/ 



TT 




TT - TTT^ = 0. 



By (2.8) and (2.9) of Lemma 2.1, (4.3) simplifies to 
(4.4) -2|(e.)-^(0.)=O, 
which is satisfied by the prior tt{(3, 9, rj) cx {9j]) ^ . 
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Finally, when 77 is the parameter of interest, we need to solve 

(4.5) ^ ^ ) N / X 

+ A L^W^^)n] - — L\] = 

From Lemma 2.1, (4.5) reduces to 

(4.6) |(,,) + _^(,.,)=0. 
Again 7r(/?, 61, 77) c>c (flry) ^ will do. 

5. Matching priors via inversion of test statistics 

One traditional way to derive frequentist confidence intervals is inversion of certain 
test statistics. The most popular such test is the likelihood ratio test. But tests 
based on Rao's score statistic or the Wald statistic are also of importance, and are 
first order equivalent (i.e., up to o(n"^/^)) to the likelihood ratio tests. We consider 
here only the likelihood ratio test. 

We begin with the general case when 9i is the parameter of interest, while 
02, ■ ■ ■ ,0p are the nuisance parameters. Let 9 = {9i, . . . , 9p), and let l{9) denote 
the usual log-likclihood. The corresponding profile log-likelihood for 9i is given 
by l*{9i) = l{9i,92{9i),...,ep{9i)), where 9]{9i) is the MLE of 9j given 9i{j = 
2, . . . ,p). Then the likelihood ratio statistic for 9i is given by 

(5.1) MLB*{0i,X)^2[l{9)~r{9i)]. 

Then from Yin and Ghosh [1-5] (also from ['>], (5.2.18)), a likelihood ratio match- 
ing prior TT is obtained by solving 

(5.2) 

^ f T-iTsuj:.r 9^^o&f ,_\ , d (j-ijdi: _^^^,^-i^, glog7 9^ log./ , 



s=2 u=2 
P P 



891 



In the present case when /? is the parameter of interest, (5.2) reduces to 
(5.3) 

^ ^rf9H9,r'A+^(v'v~'A+4-Jv'^^" 



09 y ' ' ' J dTjy ' J dp y [0(3 

r 2p//9l0g/wa2l0g/.. ^ .d^^logf. ^ ,d^\0gf. 



From Lemma 2.1, (5.3) reduces to 

-(^.) + -(,.)+, ^-(-)=0, 
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I.e., 



Again tt cx (Oif} ^ provides a solution. 

Next, if 6 is the parameter of interest, the LR matching prior tt for 6 is obtained 
by solving the differential equation 

Again from Lemma 2.1, (5.4) reduces to 



I.e., 



d .^^.diT 2 2^. ^ 



which holds for tt cx: {9t])~^ . 

Finally, when i] is the parameter of interest, the LR matching prior is obtained 
by solving 



(5.5) 



dri\ \drj \ drj drf ' ' if' 



Once again, using Lemma 2.1, (5.5) reduces to 



and the prior tt cx (07]) ^ provides a solution. 



6. Posteriors and numerically computed coverage 

The prior 7r(/ii, ^2, ^, ??) oc (0r/) ^ is improper. In this section, we write down 
the marginal posteriors for /3, 9 and 77, and discuss methods for finding the HPD 
intervals for each one of these parameters. The analytical findings of Section 4 are 
strengthened with some numerical coverage probability computations. 
The marginal posterior of /3 is given by 

7r(/3|Xi,X2) CX j 'n yi H -g^^ I drj. 



Probability matching priors 79 

Putting 77 = z[S22 + P'^Sii — 2PS12/ Sii]^^^^ in the above integral, one gets after 
simplification, 

— 1 

(6.1) 4/31X1, X,)ocfl + -^^-^^^/^")' 



S22.1 

where S'22.1 = S22 — 812'^ /Sn. This posterior is a t-distribution with location pa- 

1 /2 

rameter S'12/S'ii, scale parameter {S'22.1/ (n — 2)} and degrees of freedom n — 2. 
The posterior of 9 is given by 

(6.2) ^(0|Xi, X2) oc exp(-Sii^/2522.i'/V^^)^[e>o], 

so that has a Gamma distribution with shape parameter n — 2 and scale pa- 
rameter (S'llS'22.l)~^. 

Finally, the marginal posterior of rj is given by 

7r(r;|Xi, X2) o,rj-'/'{^ + ^S,,)-^-^'^^ 

(6.3) 

ocr;"-2(^2_^^)-("-3/2). 

Oil 

The construction of HPD credible intervals is fairly simple. The posterior of (3 
being a univariate-t (thus symmetric and unimodal), from (6.1), the 100(1 — a) % 

1 /2 

HPD credible interval for (3 is given by S'12/5'11 ± {«S'22.i/ {n — 2)} ' i„_2:Q/2i where 
tn-2;a/2 deuotcs the upper 100(a/2)% point of a Student's t-distribution with n — 2 
degrees of freedom. 

Observing that the posterior of 6 is log-concave, the 100(1 — a) % region for is 
given by [9i, O2]; where 61 and 6*2 satisfy 

(6.4) cM-Sii"^S22.i"V0i) = e2-<"-'^ ^m^s^,^'^ 822.1^ '^^2) 

and 

(6.5) / ' cxp(-Sii^/2522.i'/V^)(5ii522.i)'^de = 1 - a. 



It is important to note that \i w = ^ , then the posterior pdf of w is given by 
7r(u;|Xi,X2) cx w'''^ e^^{~wSl[^ Sl!^^). 

Noting the log-concavity of this pdf as well, the HPD region [t«i,W2] for w is 
obtained by solving 

(6.6) wi''-^ eM~wiSii^'^S22.i^'^) = W2''-^ eM-W2Sii^'^S22.i^'^) 
and 

n— 3 

(6-7) / ^ exp(-ii;Siii/2^22.i'/')(SiiS22.i)'^ dw^l-a. 

Clearly the solution [wi, u'2] of (6.6) and (6.7) is different from the solution [02^, 0^^] 
of (6.4) and (6.5). 
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Table 1 

Frequentist coverage probabilities of 95% HPD intervals 
for /3, 8 and ri when erf = I and cr| = 1 



p 


n 


13 


6 


V 


0.25 


4 


0.952 


0.947 


0.949 




8 


0.946 


0.955 


0.950 




12 


0.954 


0.952 


0.948 




16 


0.952 


0.954 


0.950 




20 


0.945 


0.948 


0.950 


0.50 


4 


0.950 


0.952 


0.949 




8 


0.944 


0.952 


0.948 




12 


0.954 


0.953 


0.944 




16 


0.946 


0.950 


0.949 




20 


0.952 


0.948 


0.949 


0.75 


4 


0.955 


0.952 


0.953 




8 


0.953 


0.948 


0.949 




12 


0.950 


0.946 


0.947 




16 


0.948 


0.946 


0.951 




20 


0.956 


0.946 


0.951 



Finally observing that the posterior of 77 in (6.3) is log-concave, the 100(1 — a) % 
HPD interval [771 , 772] for rj is obtained by solving 

<->ll Jll 

where 

c being the normalizing constant. 

Now we evaluate the frequentist coverage probability by investigating the HPD 
credible interval of the marginal posterior densities of (3, 9 and i] under our proba- 
bility matching prior for several p and n. That is to say, the frequentist coverage of 
a 100(1 — a)% HPD interval should be close to 1 — a. This is done numerically. The 
results were fairly insensitive to the choice of cti and a2- Table 1 gives numerical 
values of the frequentist coverage probabilites of 95% HPD intervals for /3, 9 and rj 

for (Ti = (72 = 1. 

The computation of these numerical values is based on simulation. In particular, 
for fixed (/ii, /i2, cr^, cr|, p) and n, we take 5,000 independent random samples of 
(Xi, X2) from the bivariate normal model. In our simulation study, we take /ii = 
fi2 ~ without loss of generality. Under the prior tt, the frequentist coverage 
probability can be estimated by the relative frequency of HPD intervals containing 
the true parameter value. An inspection of Table 1 reveals that the agreement 
between the frequentist and posterior coverage probabilities of HPD intervals is 
quite good for the probability matching prior even if n is small. 

7. Summary 

The paper considers several probability matching criteria, and develops a prior that 
meets all the matching criteria individually for several parameters of the bivariate 
normal distribution including the regression coefficient and the generalized variance. 
Future work will address development of matching priors when the parameter of 
interest is the correlation coefficient. Possible multivariate extensions will also be 
considered. 
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